>> now, to the world of law. former judges from the colorado and indiana supreme court joined at indiana university law school professor to discuss the future of law schools. this panel is part of the seventh circuit bar association annual meeting held in indianapolis in may. [applause] >> things and things from making time on the program for this conversation about the future of our profession is brainwash side, and randy shepherd could end up spending most of my life as chief justice of the indiana supreme court. he now doing senior service in the intermediate court and i have an appointment over at the indiana university mckinney school of law. to my right is for backup kourlis -- rebecca kourlis who spent more than a decade on the

colorado supreme court. she left there perhaps 10 or 11 years ago to be the founding director of the institute for the advancement of the american legal system. she does all sorts of research and programmatic work on the improvement of the court and the improvements of legal education in the legal profession. and then, profess them -- professor william henderson, after his legal education at the university of chicago became a clerk at the seventh circuit, but happily has spent his recent career at the indiana university school of law and i'd say there's nobody who's done more intriguing work about the future of our profession than bill and becky kourlis. i will begin by setting the stage.

there are many things about moderate legal education that we as lawyers know a little about, but not necessarily know about in detail. they provide considerable foundation for evaluating the current state of law schools both now and in the future. now, you could actually describe recent developments by using just a word. the eight words would be rising tuition, massive debt, fewer jobs and fewer absence. let me say a few words about each of those to begin laying the foundation for a conversation today. no mystery to anybody that tuition in higher education has been rising.

indeed, it has been rising for the lifetime of virtually all of us here at a rate that receives twice the consumer price index. it has been rising at a rate so rapid that the finance officers of the american university system has created a new measure. it is called the higher education price index designed to give a more accurate reading of how much -- but that now be called the sticker price as opposed to the nominal price. the sticker price has been going out. in the two decades that just ended in the last century in the first decade of this century, the general rate of inflation for higher ad with 71% real actual adjusted for inflation, 71%. but the raid in moscow says that

71% over that, but 317%. it has actually been rising at an average rate of 8% a year in recent decades. that of course has led to stories we all read about the largest amount of debt that's evidence, both undergraduates and of course law student experience, rising to the point for americans now own more and education debt than they owe on their homes. really quite a stunning change in the american debt picture. the average law student debt on the way out the door is about $85,000 in the public schools and about 50% higher than not, 122, 125,000 for students who have gone to private schools.

this continues of course in general to be easy to read about. just last week in "the wall street journal," a new wrinkle on debt. a federal program in place for a while but i think it's called parents pause on the front page of the second section about the opportunity parents have been given to empower their children to borrow more money if the parents would cosign and then the federal government would enter a guaranteed so that foreign money could be borrowed. it is now clear parent class is having an effect on the disposable incomes of, if you will, working middle class parent, not just children, but the collection of parent. it has become a pervasive part of what it means to be not only a university graduate, but a law graduate.

third, fewer jobs. one of the things we as lawyers didn't notice unless we had a child or a very close friend looking for work over the last two decades is that we have been turning out more jd graduate and turning out more law license holders and there were jobs for lawyers for a long time. the reason we didn't notice that unless we had somebody directly involved with it is the way in which employment for new voyeurs was reported didn't do a very good job of telling us where they were employed, simply told us via march whether they were employed and for what kind of organization they were employed. it was quite ordinary as recently as 2000 to read announcement once a year from the national association of law placement or the american bar association that was paid this

was a great year. 90% of our graduates have found work within nine months, now 10 months is the measure and not sounded like a very happy result. well, as you recall, there were a series of scandals about the way in which this was reported in one of which of course occurred next door at illinois. our friends from illinois will particularly recall. they asked tougher questions. in particular, how many law graduates, license holders, find a full-time ongoing job for which you have to have a law license. that is the hardest way of asking the question. in the last half decade or so, the answer to that has been last year was 59%. that is to say about the people who turned out of the school, how many people found ongoing work full-time for which you had to have a law license and the

answer to that is 59%. that number has stabilized. it has been as low as 57%, but the number has stabilized over the last couple of years still means that a lot of people don't find work for attorney general's offices are prosecutors offices or private law firms. the other reason is stabilized of course is that keeps declining. the way in which you calculate the percentage gets altered. the fact is that over the last five or six years, the number of jobs, full-time license required type jobs has continued to decline. as recently as five years ago it tended to be sent and like 20,000. mind you we were graduating 40,000 to 45,000 or maybe a little bit more, but there were

five years ago about 28,000 jobs, the kind most people think they are looking for when they go to law school. that number has this year fallen to thousand 700 p. a slower decline perhaps been earlier in the decade, but a pretty small number. well, college seniors and second generation aspirins in college law advisers can read those numbers and so the result has been a dramatic change in how many people apply to go to law school. within the last decade, the normal number and it was the highest numbers we'd ever seen in the country, the all-time high was 100,600 applications is a funnier number, so it seems to me more important to focus on applicants.

that number had been something like 100,000. 100,698,000 for quite a long time. it dropped dramatically in 2016 the number was 56,000. so it is on the order of a 50% decline in the number of people who come down and put in their applications. as of last week, so far this year there were 50,200 down from last year, just about 8%. that is the number i would say has probably stabilized, but there are some things happening inside the number that are particularly is. for instance, of the matriculate, the people that actually get into school, those who come into school with lsat of over 160 are half of what they were five years ago. the number who had 150 through 160 is stabilized. the number of people who comment

under 150 is up 80% from where it was. so there is a very noticeable shift in the matcher eric to use the easiest word. others might take talent or suitability for law school. the change in student legal aid as the competition for a lot applicants increases, schools have used more and more legally chasing the numbers because it helps ratings and of course students in the law school. the result of that is the amount of merit waste aid over the last decade has gone up 10%. the amount of merit aid has stumbled and there is a third category called both merit in need-based.

i haven't spoken yet to an actual law school admissions officer who didn't tell me that both men could merit. most of that goes to the people who have the best metrics. so it is the folks at the back end of the line end up with a far greater share of that debt when they lock out the door with their degree. i've been particularly worried about the effects of this, the demonstrable effects of this minority applicants because their profile in gpa and lsat isn't as good as asians and caucasians. the hispanic applicants and african-american applicants have lower metrics and it's harder for law schools to hand them scholarships or financial assistance if you will. as i said earlier, really better thought of these days is

discounting. we have some schools in the circuit, by the way, who don't charge any tuition at all for a substantial number of students. indeed, we have a few schools who sent the students money. they don't charge any tuition. they write checks to the students to support living expenses. only if you do that but it's part of this greater trend. and of course, again last week the news that the bar passage rates in the country, no great surprise have likewise been following. february is not as good of a measure as july. the february rate here in the state with the lowest we have ever experienced. on an important note, that this is not gone unnoticed by her friends and the academy. the number of people in the academy who say this too will pass and we've heard before is a shrinking number.

research done both by idols and by various entities with which professor and its associated about how we change legal education to adjust to what it looks like the profession will be is really very encouraging and very substantial and the easiest one to see if the change in what's now referred to as experiential learning. how many actual hands-on experiences do we give students has been dramatically changing over the last two decades because the schools are determined to do as good of a job as they possibly can. we are going to hear some intriguing news about all of those trends. we will start with justice kourlis. >> professor henderson and i have to stand because the monitor is only visible from up here.

not for morris dees. i apologize for breaking up our little trio and what referred to be seated. as you know, im becky kourlis. i have a background, thank you, as a trial court and appellate court judge, but now in a research institute at the university of denver that we refer to as a think to tank i else, which is the worst acronym in the world and i apologize. but we were named by a federal judge. and he wouldn't let me put justice in the name because he thought justice was a term that was to assist up to bold to the interpretation of the day. we work in four different areas and actually, i have spoken to this assembly on prior

occasions, but usually in the context of our civil justice reform work. we do work in legal education, civil justice reform. we work on the family justice i is state courts and then we work on judicial selection and performance evaluation. we've done a little bit in indiana on that topic as well. our process is we identify a problem. we gather research, put together wonderful groups of stakeholders. we create models. we then facilitate and monitor the implementation and measure outcomes. in common to tank. the problem here, which we are talking about this morning, chief justice shepard has already identified, which is in the context of legal education, we are struggling with how to prepare voyeurs for the

profession as it is evolving so that we begin to revitalize the numbers and more importantly the role of lawyers in our society. these are the drill down numbers to which he has also just referred. this is the percentage of 2015 the law graduates who did not land full-time employment requiring bar passage. bill henderson and i just touted the 2016 numbers in the number of days, so i apologize for the fact these are a year old, but this is the best set of numbers we have at the moment. 30% is the percentage of 2015 law grad who didn't land full-time employment recognizing their law degree. 25% who didn't land full-time employment including

professional and nonprofessional this is deborah risa number. 71% of third-year law students who believe they have sufficient skills to practice. this set of numbers is really aimed. 71% of the third-year law student who think that they are locked and loaded, ready to go into law practice. however, the law professors who teach them say that is 45% and, wait for it, the percentage of practitioners who believe that new lawyers have sufficient skills to crack this is 23%. so, there is a problem they are and we developed a project intended to shed some light on ways of addressing this. we created our foundation for

prop this project. we have support from the hewlett foundation. this is a three phase project. we put together an advisory committee consisting of crack tensioners, judges, justices, representatives from the national service edition of our exams from the national conference of bar presidents. the national conference of bar examiners, representatives from the public defenders office of the aba and even the attorney general of colorado. we developed a plan. the first piece of that plan is to identify the foundations that entry level lawyers need to this. the second phase will be to develop measurable models of legal education that support those foundations in the third will be to align market needs

with hiring practices in order to incentivize positive improvement. so phase one, we developed a survey based on existing research for those of you who are made to in this area, such a research as schultz and benedict and with the offense and of our advisory council, we distributed that survey through bar associations across 37 states in the country. actually, there were individuals from fifth the state to insert, but the bar associations in 37 states distributed to survey and we have a little bit of crossover people who practice in two states. we got 24,137 valid responses to the survey. if there're any of of you in

this audience who answered the survey, thank you for your time. it was a little bit at a prodigious undertaking because the survey itself is 25 minutes long. we got over 40,000 actual responses and 24,000 valid responses. they responded base nearest the legal profession. some of the numbers are not wonderful in terms of demographic, but in fact, the respondent days is representatives, both geographically and in terms of the size of firms in which lawyers across the country practice, the amount of money they make and the demographics themselves. we asked about 147 different foundations in the survey. we categorized the foundation

and they broadly fall into three different categories, but some of the examples are identified relevant facts and legal issues, critically evaluate argument, conduct and defend, listen attentively and respectfully, make decisions, work as part of a team. the three categories into which we divided those foundations are as follows. skills, characteristics and competencies. so, remember we have 147 foundations and we are dividing them into these three categories 27% of the foundations were skill base. 28% characteristic-based and 45%

competencies. we also graduated the questions that we asked of the respondents. we asked what is necessary in the short term. we asked what is advantageous but not necessary in the short term. we asked what is necessary over time and we asked what is not relevant at all. the screen right here doesn't think i'm great, so it looks like i'm missing a cylinder in the middle, but fortunately not on your screen. so here are the results. there were 77 foundations identified as necessary in the short term by 50% or more of the respondents.

so again, 147 foundations, 77 were identified as necessary in the short term by 50% more of the respondents. but what emerged surprised us. we call it the character quotient. here it is. out of the top 20 of those foundations identified as necessary in the short term, nine are characteristic, 10 are competencies and one is a legal skill. there it is. the 10 competencies, keep confidentiality arrived on time. this is a millennial issue, right? arrive on time. treat others with courtesy and

respect. listen intently and respectfully, respond promptly, take individual responsibility. emotional regulation and self-control. speak professionally. write professionally, exhibit tact and diplomacy. in the characteristics, the other nine honor, commitment, integrity, trustworthiness, diligence, strong work ethic, attention to detail, conscientious, intelligence and a strong moral compass. so cumulatively, we refer to this as the whole lawyer. but what about legal skills? this is extensively why we go to law school, right? the legal skills show up in them must be acquired over time set of responses.

these 20 characteristics are the top 20 for these foundations are the top 20 that must be acquired over time according to our responded base. and here they are. develop appropriate risk mitigation strategies. provide quality in court trial advocacy. cannot and defend appeal, prepare for participated mediation. all of those skills that i suspect all of us would identify either on the transactional or the litigation side of crack this as being necessary. but the take away here is we expect people to acquire these over time. we don't necessarily expect them to show up at the time they sign up to work on a ready to do these things are qualified to do these things. so now here is the second part of our survey and here is where

it becomes a little more relevant to all of you. in the second part of the survey, we asked respondents to consider the helpfulness of a set of hiring criteria in determining whether a candidate has the foundations they identified as important. so, you have to understand, we took as a given the fact they had already answered the survey and identified what they felt was important and then we said, okay, how would you hire in order to suss out the individuals who have those foundations. we did not ask them how they currently higher. and he was poor oliver barrett before it who went to yale and harvard and was no senator of the log review and did a summer

clerkship for welfare. so, the avowed notion is that this person would get an interview pretty much anyplace in the country, but does this person really have on his or her, in this instance is indicia that would suggest that he has those foundations that people said they were looking for. so, what would people focus on if they wanted to hire the whole voyeur with a character quotient? we identified 17 criteria and we asked the respondents to rate them from very helpful to very unhelpful. so take a look at the 17 criteria quickly and then i will show you how quickly the results worked out. here they are.

top of the list, with the goal employment, recommendations from crack to sharon's or judges. bottom of the list, log review in journal experience. now, this gradation is all respondents. you will note that this verse eight have to do with experience. what the respondents told us is that the character quotient for the whole voyeur metric emerges through experience. we broke out the data by the firm government employer legal services with firms of 100 plus class ring and law school move up with firms that two to 100, there is a little bit of a shift

in the top eight particularly the second floor, but it's still the same top eight. in smaller firms or solo part titian or settings, specialized classes, specialized law school classes move up, which makes sense because there is a little bit more demand for actual ability to jump in on a project or topic. government employment, and the top is still the same, a little bit of shift in priorities and then legal services or the public defenders office. again, you see a particular law school course specializing presumably in a criminal area moves up. so what does all of this tell us as we are thinking about the future of legal education and the future of employment of

voyeurs? first, it tells us that practice ready may not need the best term to define what we expect from their lawyers entering the professions. we all expect new lawyers to learn on the job. but we do expect them to emerge from law school as whole lawyers, with a blend of legal skills, professional competencies and the character quotient. when it comes to evaluating whether a new lawyer has that character quotient or that land of foundations that we have labeled whole voyeur experience really matters. to date, the foundations for this project has put out two reports, both of which are available on our website.

the first really focuses on the evaluation of the data in terms of what respondents are looking for a new lawyers. that is where the lawyer in the character quotient emerged. the second report focuses on hiring the whole voyeur and experiential component. we are now moving on to the measurable models of legal education that support these foundation, working both with the law schools and employers to shape ways whereby employers can recognize on a law school resume and other forms that they have the individuals they have identified do not give credit where credit is due.

the practice project at isles and indeed is the name by allie kirkman, whom some of you may know or have heard speak. she was supposed to speak today in a medical circumstance kept her away from you. so i just want to note that she's the one who really developed this project in her information now appears before you. so i look forward to the discussion about this and about what it means for the evolution of law school and of the procession. [applause] >> well, that was a very good pinchhitting job by judge kourlis. i've learned a lot from the session and am very grateful to be back at the seventh circuit, giving a talk to a court that i

used a clerk on. the point i'm going to go through today are the central pieces i want to advance that we need to look at the future law school is going to drive from the future of law history of fall practice is changing pretty dramatically. i think it is also had a fairly long period of time is kind of like the boy and frog metaphor, for those of you that have heard it. put a frog in the water and heated up and and it will just die there. but if you toss it in the hot water, it will jump out immediately. to a certain extent, that's a lot of what's going on with millennialist today. for a certain extent, those of us in engaging bar, we lack that fresh perspective. the price of oil is changing very, very dramatically. i'll give you a little bit of an overview and maybe a couple of prescriptive remarks. go in on this idea that judge sheppard talked about what is

going on in law schools, here is some data that i was able to calculate through data posted on the aba website. you can see the average entering class-size for about four decades, for those of you who were in 1971 quite a while ago, the average class size was 246 and it hovered in that of four over 40 years. a high water mark in 2010 of 262 and drop precipitously to the last four years a small entering class-size. what's been going on in the background we've been adding more law schools since 1971. 147 law schools in 1871. we had a high water mark of 204. /they were down to 201, but that is a tremendous amount of fixed costs spread over a larger

number of law schools. as you imagine, a hotel or airline or restaurant chain that lost 31% of its pickup resolution between 282 and 182, you see a massive consolidation that was taking place in order to regain some pricing power. universities don't think that way and so we are kind of moping along, trying to figure out what the next move is going to be. i want to tie this into the labor market is somewhat. one more piece of labor market. this is the number of aba graduates from 1973 to 2015. we extrapolate what the graduating classes are going to be by the size of entering classes and you can see those are a bright green bars showing we are going to levels of law school graduates that we haven't

seen since the 1970s. this seems to be far beyond cyclical. it certainly could be structural and a lot of people look at the data without taking too much deeper then eventually this will rejuvenate the kazoo drop-down to a lower number and everything will be fine. i think it's actually the nature of law practice changes in such a way we can't just think on a smaller kind of base number and kind of build back up. someone alluded to a little bit in the report that is given by the chief administrator of the seventh circuit. so, to look at this from an holistic perspective i get data from the u.s. census bureau and look at the total market for legal service in the united states. about $290 billion. these are enterprises or business is providing legal services. as a wonderful classification by

the census bureau called class of customer data you can drive how much you are selling, how much or provision of legal services to individuals and how much you assign to organizations. you can see a big split of about $70 billion to individuals and the lion share of it going to organizations. there is a famous study, he'll study, albee study, at least in a semi circle called the chicago lawyers that he that was done -- published in the early 80s from a sample drawn in 1975 of the chicago bar. the finding came out of this, which was if you want to understand the structure of the bar, find out who your clients are. from your clients you can pretty much tell a lot about who the lawyer was. this is the two hemispheres. because roughly half of individuals than half the bar were serving organizations. i replicated that study the resample in 1995 and they found

out the lawyers that were serving organizations enjoyed a nervous in-house lawyers, large firm lawyers and the small firm and solo portion of the bar serving individuals was much more economically distressing. this is going back over 20 years now and almost certainly the trend or pattern has continued forward. this has really big implications for law school because they show up with an idea on pop culture. the statistics are pretty overwhelming about the number of individuals would like to vindicate their matters and get them full justice. they just don't have the capability to hire competent legal representation. so, this is something that just came to my attention a few weeks ago because i was working with judge kourlis in november.

they put together this report and here is a couple really shocking statistics. i can remember sitting there saying i can't believe the numbers i'm hearing. the median judgment is $2400 in the state courts from a civil civil manner. 2400 bucks. it is making it really hard for me to make a living from that. .. it's just continuing to get more exacerbated to a point where it's really pretty close to a breaking point. this is not the type of legal system by which lawyers can earn a limit. it looks pretty close to a

breakdown. you can get a lot of lawyers in the big cases but for the typical case it really doesn't work for the lawyer. so i want to, i queued up this theme of the segment of the bar that serves individuals and organizations. there's this concept put forward there's a late legal demand. there's individuals that would like to be up to solve their legal problem if there where they have one but they can't afford a lawyer so there's these technology products that are coming online by ways they can engage some self-help. y combinator is a famous facilitator. this news report talks about 11 legal tech companies being fairly excited, being financed by the y combinator and the y combinator leadership pretty bullish on the legal sector. then there's this quote that

pulls out of it the basically says it's in the individual market where financing it, that the legal tech startup pretty reticent to commit dollars to the large law firm segment. for variety of different reasons it's difficult to sell efficiency to large law firms and that's what sam altmann was getting about. on the one hand, we have these kickstarter said beginning to fill the void, the justice gap by the inability of the civil court system to accommodate, to be able to marry up practicing lawyers in cases in way to make the economics works. this is a sea change and it wouldn't call all practice. i would call it the legal industry. if we go back to thi the chart i drew. now i'm breaking it down by total amount of spin. on the winning of the spectrum the nose at the bottom, $220 is your average legal budget for the individual in the united states.

if one of us gets divorced we will wipe that out for several hundred people probably. it's an inbound system, is not a lot of money but if you look to see over half of the dollar spirit are spent for the basis of organizations. i calculate whether average budget would be. over 50% has 2 million or more to spend on legal services. this ithis is a list of companiu can get on a book. it would be shorter than the yellow pages and these are the folks that spend a lot of money on legal services. this group of folks are now in a position where they're complaining about the cost of legal services and they are beginning to look for alternatives to traditional law firm practice. and so here's an article that comes from the "wall street journal," talks about law firms facing the competition. they're all clients are in sourcing legal services.

euros another clip to come some jpmorgan come from bluebird on jpmorgan basically hiring a bunch of developers to develop software to automate jobs that would or no to be done by lloyd picked as much because lawyers are too expensive. there's a way the lawyers could get through the amount of work that needs to be done so they are hiring ai enabled developers to develop tools to provision of their own legal services. this is a trend that is just to take off the ground. so here's a picture of what's happening in law practice today. you can see this is three trend lines going back to 1997 and it shows against the 199 the 1997 benchmark, the first year the system up and running comes with 20 years i can go from the baseline. you can see the change in employment of employed lawyers,

w-2 lawyers in law from sector come in the government sector and in the enough sector. the big lead from this is an house has been growing dramatically. grew from 34,000 in 1997, to over 105,000. there's as many lawyers in house today as there are that employed domestically. it's pretty staggering and it is growing, the hiring rate is about 7 seven and a half times more lawyers being hired in the enough legal department that are being hired into law from sector. this should surprise folks, we just ignore the top light and look at the bottom to find. government side almost twice as many lawyers than the private sector, at least since 2006. i don't think the government feels like it's been on a hiring boom at all. this is telling you something big and structural is taking place in law practice. basically the corporations

between building carbine, they're choosing to build. so having thought about this topic for quite a period of time and doing more field research as opposed to just reading, spend a lot of time with in-house legal departments trying to understand the changing buying patterns. i came up with this, and the real take away from this is what i call the type six client to the far right. this is, these are sophisticated law firms embedded inside major corporations that have the ftes in the specialization, the budgets arrival almost any law firm in the country and to a certain extent they made a decision to build this capacity enough because they just can't buy what they need externally in the market. there's a right of reasons why law firms transition or failing to transition to fill this gap of probably the biggest one is

that magic of billable hour. it's how we measure production, how we sell our services and how twe compensate our lawyers and that's antithetical efficiency in the more for less mantra we are from in-house legal departments. these folks taking matters into their own hands. the most significant segment is probably the legal operations folks which is the gold boxes to the right. these are very sophisticated folks that are tied to data processing technology to people system to drive up quality and to try down costs. there are also being aided and abetted by the green bar which is the procurement folks and using all sorts of systems to rapidly turn a portion of legal services market into commodities. this ithis is a big seachange at affects ultimately who we hire from law school and the skills they need to succeed. can you raise your hand if

you've ever heard of clock? really? nobody has heard of clock? okay, one hand. corporate legal operations consortium. this is the fastest-growing, this is big and growing very fast. they got their convention next week in las vegas. this is the fortune 500. this is the professionals that are in gold that he getting together to compare notes to build system to provision legal services faster for the internal clients. this convention las vegas will probably have 1500 people that are in attendance. every possible vendor in the legal ecosystem will be there, and basically they rapidly figuring out ways to drive down their costs. i would say in the way it keeps them pretty far afield from civil disputes.

this is rapidly changing. again, driving down the demand for legal services and to a certain extent, a change in the nature practice in a very, very fundamental way. i hope that a few more people have heard of richard. raise your hands. we have quite a few people that have heard of richard. richard wrote a book called the end of lawyers in 2008 and had the question mark on the end which was meant to signal that what's the purpose of lawyers. he wasn't making a disclosure statement please what's the purpose of the lawyers and it's to serve clients in the state and ultimately tribal living after we had served those two duties pretty thought challenges to rethink the nature of legal services and he wrote another book in 2013 called tomorrow's lawyers which i committed anybody. it's only about $20, it's 160 pages. you can you can buy a copy, read it on a plane flight between here and the east or the west

coast. it's absolutely marvelous accessible book. richard has been writing for long period of time at a fairly high level of abstraction but now in my fieldwork i see more and more evidence of what richard is talking about is coming into fruition. a model that richard uses, and there i'm going to get, speak to the future of legal education because it's the future what the market is telling us it wants to either cloc or start of ecosystems to tap in to latent demand for individual legal services. when we think about legal services, richard is making a claim everything can travel along this five stage progression. the spoke is tailor-made and is what we learned to do in law school. tailor-made in argument to win in court. tailor-made a contract the consummate the desired transaction. so we can all get what bespoke is what he's talking that moving

to standardized to systematize to packaged to commoditized. any lawyer including me would agree that commoditized is a fairly frightening place because it means it may be free over the internet and it's hard to make a decent living if our work product is going to be free over the internet. a lot of lawyers retreat back to this idea adjusting the most technically sophisticated stuff, the brain surgery learn, we claim we know how to do with learned in law school are learned over a period of decades in practice. richard is making a claim that the systematized packages with the future of law is going to reside. if you think about the individual corporate hemisphere i talked about before, what is it struggling from? lack of lower productivity. this inability partially or substantially perhaps a function of how the legal system is set up to its procedures and its

trial practice in the presumptions that are built into it. makes it just uneconomical to access legal services through the standard format t through te court system. and then from the corporate perspective for such a massive amount of need they can't afford to just pay large firms to continue to do that. they need efficiency so they're doing it themselves. the systematized package i think salz the problems, the lower productivity problems on both sides of the question. what's the human d-uppercase-letter? information systems, systems and digitally, finance marketing, and law. of course you need law but these are other disciplines that are not traditionally taught in law school. it would be beneficial to have an eye drop of those disciplines within the legal education but the future is the ability to communicate across those

disciplines, to effectively engage in chains of multidisciplinary fractures and come up with solutions that really are outside the domain of legal expertise that require legal expertise for them to ultimately serve the best interests of clients. that's a pretty tall order but i see every time i log into the cloc server i see the stuff that's getting talked about, i see the future pretty clearly comes across from what those professionals are talking about is definitely moving in this direction. i want to end up here with this idea of people and process and technology. some variants of that that we will need to incorporate into legal education. it's going to happen slowly, much, much too slow for my satisfaction but we will have to work backwards from the demands of the marketplace. you guys are the folks ultimately they can put pressure on the law schools to change

what we teach and how we teach. i'm going to do one quick exercise and then we will be done. this is a famous by my life, famous study done by scholz and ascetic, the people make the lsa secret is published in 2008 but all the data was gathered in the early 2000 and basically they did a study that was somewhat akin to what the judge did. they tried to delineate the factors you need to be successful as a lawyer and it was based upon the university of california berkeley alumni base and also university of california at hastings alumni base, about 1200 lawyers in the sample. after they delineated these different 26 factors using gold standard psychology methods then they came up with behaviorally anchored reading skills to judge the quality of the lawyers, how

they performed on these 26 different dimensions. if i was in the sample, i would have a score for my supervisor, my peers and my negotiation skills or my creativity and innovation or my practical judgment. they took the data and call it a back academic predictors, the undergraduate gpa that are often prized on admissions. i will put this question to you and let me tell you, let me see if you can get some of the takeaways from this study. which at the beach below with a better predictors of lower effectiveness, -- which was it? was it a or was it be? b? everyone says b. you got it right. how does your law firm or how do your chambers higher?

do you rely on b or a? you rely on b? i suspect my experience has been we rely quite a bit on a even though we know actually b is the right way how people flourish actually in practice. i would exhort you to come if the bar, and happy to help on this one, we can think deeper about how the practice of law is changing and the kind of professional skills that we need from entering associates that we can iterator dialogue and agree what's price and that's how we're going to hire. i think we can break at a some bad habits and begin to are some really great people and change their curricula in ways that serve our society. i thank you very much for your time. [applause] >> we persuaded brian welch to preside over q&a and comments,

and all of us are glad to respond to anything that you heard in your the three of us say. >> i don't see any raised hands. do we have any questions? were going to try to walk a microphone around the people can hear the question. judge barker, you have the floor. >> good morning. is this on? going back to your last slide, bill henderson, the a and the b, is a reflective of the criteria for law schools use in choosing law students? >> guilty as charged. [laughing] yeah, i say to myself judge a party gets up come she's going to throw us a fastball. i was right about that. the law schools, we feel like we are locked into a kind of a positional competition through "u.s. news & world report."

we do not for undergraduate credentials to admit on and that's how we reward scholarship money. writ large this is how most of the 200 schools, aba accredited schools do that. maybe some of our self regulated organizations can get together and figure out ways to break the logjam. clearly there's all sorts of deleterious consequences on society and to think we will learn, will come to regret where we are at. having been on the inside of law schools, not only at my school but talking to a lot of other law schools, it's a very serious issue to kind of flout the convention of use news and world report. your alumni and your students and applicants all can be easily turn into a tailspin if you dropped ten points in the rankings. so i'm a believer, unfortunately

an rankings management, but there are ways to come i think the law schools themselves overweight lsat and there's ways, i know one school that actually folks in employment outcomes and admits people with lower lsat and lower gpas because they know they are more employable on the back in and they run the math. that's very early in where we're at here and i would love to break out of that, that present dilemma. >> almost impossible for an individual school to do that because you get killed in the marketplace, changing the discounting, for example, to be a near fatal. deborah jones merit from the ohio state university school of law has done some important work i think trying to tease out a few the options that might exist for collaborative work, and she's gained some attention

rightly because of that. >> we had another question here in the middle. >> i know what it lsat is an gpa but i don't know the b category. does anybody that you know of in any school or any entity use that? >> i was part of applied research company for six years and we helped law firms utilize that. let me stay centered on the shultz and zedeck study. they use a variety of different instruments, the i/o confidential organization also psychology gold standard. they use hogan products and so they used assessment called the hogan personality inventory and then another one called the developmental survey instrument. they were looking for the positive attributes and the negative attributes for on-the-job performance. they have been validated over a

long period of time and many different professional settings. larry richards is somebody was in the law firm space, he's a psychologist that has a big practice built up around the hogan products because they are excellent. he probably has tens of thousands of law firm partners, facially law firm leaders. there are folks out there who are using them. there's also other tests that were in the shultz and zedeck, situational judgment, biographical inventories which are kind of like a money ball type instrument here all of these things predicted way better than lsat and gpas. some of the academic predictors were negatively correlated with effectiveness factors. so hire the lsat a slightly lower your ability to develop business or the higher your undergrad gpa, the lower your practical judgment. and so there's a huge amount of

effort that can be put in from a law firm to begin to think thoughtfully about whose successful in the practice and why. you can't build self reflection, fortunately, and i think the legal practice dramatically underestimates, and to invest in kind of thinking through lower development. but there's tremendous research that's been done out there and happy to go off-line and equate somebody with where to begin on that inquiry but those tests are binges. >> bill, talk a little bit abot the role the bar exam plays in this process, the correlation between bar exam passage and lsat and extent to which we have built-in sort of our own piece of this puzzle that we may need to unravel. >> you have. -- yeah. i'm not sure i exactly know where you want me to go with this but i will go with what i

know. the lsat is a pretty good predictor of ultimate bar passage, but something that is about six times more effective is your law school grades. your law school grade should be a function of your lsat but when you put into a big statistical model you can measure the effect of one versus the other hear your law school grades are about six times more important in ultimate bar passage. so independent of lsat what predicts come what would predict your law school grades? motivation, all the character factors that the judge was referring to, kind of showing up. i always would take attendance in my class and i'm always struck every i get the same results. if you come to fewe fuel class u get a lower grade. i think age in applying grading in some of those people have high lsat scores.

i guess, or i should ask judge kourlis. is that the point that you trying to directly toward? >> i just wanted to raise the question, it's easy to say that this is all u.s. news and world reports fault and that there's nothing we can do about it. but in point of fact there are things we can do. we talked about collective action by law schools. we talked about changes in hiring practices that therefore incentivize law schools to produce different products. or alternatively to create different measures for the students who are graduating. and then there's the bar exam over which we do have a significant amount of control, questions about whether we should be thinking about changing the bar exam, asking different things, directing different kind of skill set development through the bar exam

is the issue that i was trying to put on the floor. >> in new hampshire they actually ran a pilot program, i will get the name quite right although there's a report that was put out by either, the franklin pierce report that basically allowed law school graduates in new hampshire to completely opt out of the new hampshire bar if they actually took a series of courses that were a simulation that allows you to build up a portfolio. the take away from the employers was the end people went to the program was just demonstrably better on so many different dimensions which i think goes to judge kourlis point, if, that initial demand from the bar on who gets admitted and on what criteria come if we change that we could end up with people that are better prepared and more effective for the inserts a client. >> the early days of that option, an a relatively small

number of students chose to do because they thought it would be harder than taking the test. that's really, i think it would be some interest in altering the final entry, if it were something valid and reliable. i remember being in a drug court setting one day where a lurking in on behalf of his client and he said my client is want to doo drug court turkey would rather go to prison. what he really meant was, drug court is pretty darn tough. it would be easier somebody to stand two years in a jail somewhere than would be to go to prison. they key would be what you want to measure, but b, can you create a measurement that actuals people accountable and separates of the people that we shouldn't turn loose on clients from the ones that it's okay to turn over important things to. >> other questions? judge?

>> i want to go back to the slide about change and replacing people with technology, and there's a lot of that going on. [inaudible] so i want to know why we are not ultimately talking about shrinkage of capacity in law schools, fewer law schools, fewer lawyers over the long run. because these jobs are not going to come back. the document reviewer jobs are not going to come back and they really probably shouldn't. >> so should we repeat the question? >> so the question is why is it really what you're talking about just shrinkage in the size of the law school, the legal profession, which is some points going to meet we don't need 200 law school. maybe we need 150 law schools. maybe we should watch that curve go down further. >> i agree with you, the

question is are we going to do it through some sort of a regulatory force or do it to the market. through the market it will be a long, slow, hateful process and then if we do it regulatory relief, which law schools closed? i completely agree with you. i think we are clearly headed down that road. we have a lot less people making cars in the united states then we did 20 years ago. >> i just want to say this is not a problem unique to law school. the federal trade commission has been studies about how industries should, the word that you sometimes is rationalize come in the face of declining demand. if your horse and buggy industry, demand is just declining and so there's a big debate at least in antitrust circles about whether you really should just let the market do this because of the a better result ultimately even if painful along the way rather than from the top down.

>> i would add just one other consideration, and bill alluded to rebecca is work on the justice gap, the unmet legal need in the united states is burgeoning. you can see that from the self represented litigant numbers, the pro se litigant numbers. so yes, i think part of it is downsizing of the profession but i think there's another component here about the profession serving the people who need legal assistance, and doing it in innovative ways that are affordable, unbundled legal services is an example. maybe we develop things like limited license, legal technician options such as the state of washington and now utah. maybe we become more involved in the kinds of online services that are being provided and

playing a role. then julian henderson would take -- >> hatfield. >> hatfield. we take the position that there is a unique role for lawyers in the kinds of systems that are being built through ai, autonomous cars, for example. thinking through how to build out the rules for what autonomous cars do or don't do. so i think there is a demand for legal services, that is huge. it's a question of how we're going to meet that demand and whether we're actually going to come up to the mark in changing the way that we think about the product of what a lawyer does and had a lawyer is trained. >> wonder the things you can say is the number of law schools are shrinking. i was very surprised last week. i spent some time in the fall 2016 matriculation numbers and there are a lot of law schools

that matriculated fewer than one of people. there were some that were sus 60 or 70. the justice department as you might know has taken a position in the collective action aimed at shrinking number of competitors is frowned upon to say the least. [laughing] but one of the things that has happened is that historically on most of these graphs the law schools sent money into the central administration budget of the universities, and now it's quite common to have come my has to flow in the opposite direction. it's very difficult time to be a law dean in an american law school, and so the number of people to find employment as legal teachers has been shrinking, at the number, the amount of money that costs people like whittier for example, to say we choose no longer to subsidize this operation historically the subsidies ran back into central administration for departments

which there was not much market demand. >> who is paying for this right now? it's all federally financed and the u.s. taxpayer is a guarantee or on this one. i think that should affect the antitrust analysis somewhat. >> other questions? >> i was wondering, professor henderson, if you could talk more about the studies are trying to correlate essentially young lawyers, attributes weathers lsat or noncognitive skills with career success. how do studies measure our success? i assume as you say the highest paying jobs are using the wrong metrics, using lsat and those kind of metrics with a higher than if you just measure, say, lifetime career earnings you'll find strong correlations between lsat score and that kind of metric. how are you defining effectiveness of our success in

this kind of studies were your finding stronger correlations with things like conscientiousness and timeliness and those kinds of trait? >> you have to back out from the analysis the initial effect that it might have on your first job, your first job will affect your employment, you know, your lifetime earnings. specifically, going back to the berkeley study, the shultz and zedeck study, what they did is they took fifth 2 the 26 effects factors and they through focus groups develop descriptions on a positive and negative side for what say seeing the world through the eyes of others look like, or the ability to develop business or integrity and what it looked like positively and negatively, and then through a series of iterations were able, where they asked practitioners

to place those along a continuum, to give them a specific score. the effect of this was that the rating scales were very, very reliable and valid for predicting future performance, or predicting performance. and so that's how the berkeley study was conducted here they just took those behavioral ranking skills to supervise and employers, thick of the scores and the correlate with lsat and there was one of six of the 26 factors that were positively correlated, three of them were negative. in my own study when you look to see at how elite schools do in terms of persistence toward partnership, they get oversampled at the beginning and in the persistence of elite law school graduates, ultimately persist into partnership is a lot lower than regional law school graduates. that's probably because the fact

that law firms so preference top ten or top 14 law schools. so they kind of look at evidence that ma maybe the person but doesn't have the motivation or the characteristics to succeed long-term. i think that is rooted in a historical bias in the bar that privileges academic credentials and extrapolates way too far from academic credentials. to be an effective lawyer it's just a much bigger toolset that academic ability. just the hiring bar and even the judiciary, quite frankly, just over weights academics. you play it over a lifetime and academics but but not nearly as much as we think. >> i think we have time for one more. >> i wanted to talk about law school curricula for a moment. judge kourlis service of one of the things i felt along temple as a hiring partner in a law firm as was hiring law clerks now, that is things like long

review, you got the top of the top members of each law school class turnin in themselves writr long review and learning a skill that is frankly useless. i have not found it correlate with good writers. they learn how to write for long review. they don't learn how to write briefs. they don't learn how to write as lawyers. i'm wondering if the law schools have looked at focusing on what really helps them learn how to be a good lawyer and a good writer? >> sounds like they should. >> go ahead. you can answer, thank you. >> so the answer is there are some pretty innovative things and law schools around the country that are directed toward trying to produce better writers and better thinkers. the problem with it from my perspective is it tends to be the innovation tends to be concentrated in the bottom two quartiles of law schools around the country.

the top-ranked law school, "u.s. news & world report," doesn't see the need to do those sorts of things. they have higher employment rates as it is and they sort of our resting on their laurels. so from our perspective there are some really innovative wonderful law schools, but most of them are the ones that are struggling with their market share, which is not the right message either. >> alright. i think we will leave it there. thanks to her panel very much for this mornings presentation. [applause] >> with a 15 minute break on the schedule. he's come back in 15. i neglected to say to you that there is a wi-fi available if you want, the password is seventh circuit. should be on the slips on your table. we will see you in 15 minutes. thank you.

>> president donald trump's chief strategist steve bannon says there is no military solution to the threat posed by north korea. his remarks, as president trump recently pledged to meet north korea with fire and fury for its threats. meanwhile vice president pence is ending his visit to south america early and flying back to washington, d.c., to attend a a meeting tomorrow morning at camp david. that is expected to focus on north korea. >> tonight on c-span at 8 p.m. eastern, an in-depth look at the opioid epidemic including ohio attorney general who assume several drug companies for the marketing of opioid painkillers. >> what is different about this drug problem we have is how pervasive it is. it is absolutely everywhere. it is in our smallest communities, inner cities, it's in our most affluent suburbs. >> and friday at 8 p.m. a profile interview with secretary of health and human services tom price.

>> i think that my passion for trying to help people and my passion for a healthy society, this just feels like the culmination of a lice work. the 20 plus years in clinical practice i had turned for patients which was incredibly rewarding, overlapped with the 20 plus years that he had in the representative lifeboat of the state senate and in congress, and to have the opportunity now this time, this pivotal time in our nation's history and the system that we have to lead this remarkable department is as appalling as a big. >> phone at 8:30 p.m. buy a conversation with supreme court justice elena kagan. >> you said at the beginning of our conversation we are not a pure democracy that we are a constitutional democracy, and that said the judiciary had an important role to play in policing the boundaries of all the other branches. that can make the judiciary and

unpopular set of people when they say to a governor or a president or congress, no, you can't do that because it's just not within your constitutional powers. >> watch on c-span and c-span.org and listen using the free c-span radio app. >> back-to-school season is your which means teenagers are hitting the road. to make sure young drivers are safe behind the wheel the family, career and community leaders of america announced its keynote safety assessment initiative at the national press club. this discussion is 40 minutes. >> welcome everyone. good morning. less traffic today, yes? not for me but for most people i'm sure. on behalf of family, career and community leaders of america, fccla, please welcome all of you